Image processing apparatus, image processing method, computer program, and storage medium

ABSTRACT

Image data and image-capturing-condition information obtained by analyzing the image data are input from an external apparatus. Based on the input image-capturing-condition information, a range of angles or sizes employed in a process of detecting a specific area from the image data is determined. The specific area is detected based on the determined range of angles or sizes.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus, an imageprocessing method, a computer program, and a storage medium, adapted foruse in a situation in which a similar detection process is performed inboth an image capturing apparatus and an image processing apparatus, andconfigured to detect a specific area based on information detected in adetection process performed by the image capturing apparatus.

2. Description of the Related Art

In recent years, many image capturing apparatuses such as digitalcameras or digital video cameras have become available which have acapability of detecting a subject such as a face or a total figure of aperson in adjustment of an image capturing condition such as a focallength for the subject or an exposure condition, and many printingapparatuses such as a printer have become available which have acapability of performing similar processing to that performed by theimage capturing apparatus to detect a subject such as a face or a personincluded in an input image thereby allowing a correction process to beperformed depending on the subject.

For example, Japanese Patent Laid-Open No. 2004-207987 discloses aprinting apparatus configured to correct an image based onimage-capturing-condition information representing an image capturingcondition employed by a digital camera. In the image processing methoddisclosed in Japanese Patent Laid-Open No. 2004-207987, image data of animage of a subject captured by a digital camera and person coordinateinformation indicating the location and the size of a person imagerelative to the total image are stored in a memory card. In a printer,the image data and the person coordinate information are read from thememory card. Based on the person coordinate information read from thememory card, the person data is corrected by performing enlargement,reduction, and/or replacement such that the image of the person has adesired size and position in a specified total image area, and printingis performed according to the corrected data.

Japanese Patent Laid-Open No. 2005-267512 discloses a method in which,in a red-eye detection process, an area, a direction, and a size in aface detection process are limited thereby achieving an increase inred-eye detection speed. In this method disclosed in Japanese PatentLaid-Open No. 2005-267512, a red eye candidate area is set as a specificarea candidate, and a face detection is performed in the specific areacandidate. If a face is detected, information indicating a facedirection and a face size is stored as face information. When a facedetection process is performed for a next specific area candidate, thedirection and the size in the face detection are limited based on thestored face information. The limitation on the direction and the size inthe face detection process makes it possible to perform the detectionprocess in a short time, and leads to a reduction in probability ofmaking an erroneous detection.

However, in the technique disclosed in Japanese Patent Laid-Open No.2004-207987, the face detection process capability is not implemented inboth a digital camera and a printer, but only in the digital camera.When an image of a subject of a person is captured by the digitalcamera, there is a possibility that movement of the subject occurs afterthe subject is detected and the focal length is adjusted. This can causea difference to occur between the coordinate position of the subject inactually captured image data and the detected coordinate positionindicated by the person coordinate information. In the case where thereis a difference between the coordinate position indicated by the personcoordinate information and the coordinate position in the actuallycaptured image data, if the printer uses the person coordinateinformation produced by the digital camera, then a position error occursalso in corrected image data.

In the image processing method disclosed in Japanese Patent Laid-OpenNo. 2005-267512, the direction and the size in the face detectionperformed by the detection apparatus are limited based on the storedface information. However, the face detection capability is notimplemented in both the digital camera and the printer. Therefore, inthe technique disclosed in Japanese Patent Laid-Open No. 2005-267512,the printer is not capable of detecting the face area using the faceinformation produced in the face detection process performed by theimage capturing apparatus.

In view of the above, the present invention provides a technique to, ina case where a capability of detecting a specific area is implemented inboth an external apparatus and an image processing apparatus, perform aspecific area detection process in the image processing apparatus basedon information detected in a detection process performed by the externalapparatus thereby achieving a reduction in processing time and anincrease in performance of the image processing apparatus.

SUMMARY OF THE INVENTION

To solve the above-described problems with the conventional techniques,the present invention provides an image processing apparatus includingan input unit configured to input, from an external apparatus, imagedata and image-capturing-condition information obtained by analyzing theimage data, a determining unit configured to determine an angle or asize of a template used in detecting a specific area from the imagedata, based on the image-capturing-condition information input via theinput unit, and a detection unit configured to detect the specific areaby comparing the image data with the template having the angle or thesize determined by the determining unit.

In this image processing apparatus according to the present invention,in the situation in which the capability of detecting the specific areais implemented in both the external apparatus and the image processingapparatus, the image processing apparatus performs a specific areadetection process based on information detected in the detection processperformed by the external apparatus thereby achieving a reduction inprocessing time and an increase in performance of the image processingapparatus.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention and,together with the description, serve to explain the principles of theinvention.

FIG. 1 is a schematic diagram illustrating an image processing apparatusincluding various processing units that operate according to anembodiment of the present invention.

FIG. 2 is a flow chart illustrating an operation procedure performed byan image processing apparatus according to an embodiment of the presentinvention.

FIG. 3 is a diagram illustrating a data structure of an Exif file.

FIG. 4 is a diagram illustrating a concept of a detector for performingface detection.

FIG. 5 is a diagram illustrating a concept of a face detection processin which a detector is used in eight directions.

FIG. 6 is a diagram illustrating a concept of setting performed by aprocess setting unit in terms of directions of a detector employed in aface detection process performed by a detection processing unit.

FIG. 7 is a diagram illustrating a concept of changing the size of adetector used in a face detection process.

FIG. 8 is a block diagram illustrating an example of an internalhardware configuration of a printing apparatus according to anembodiment of the present invention.

FIG. 9 is a diagram illustrating an example of a connection environmentof a printing apparatus according to an embodiment of the presentinvention.

FIG. 10 is a diagram illustrating an example of a connection environmentof a printing apparatus according to an embodiment of the presentinvention.

DESCRIPTION OF THE EMBODIMENTS

Image processing according to an embodiment of the present invention isdescribed below with reference to accompanying drawings.

First Embodiment

FIG. 8 illustrates an example of a hardware configuration of a printer101 serving as an image processing apparatus according to an embodimentof the present invention. The printer 101 includes therein a CPU 102, aROM 103, a RAM 104, and a printer engine 105 for performing a printingprocess. In recent years, a multifunction peripheral having a printerfunction, scanner function, etc., has become commercially available.Thus, the image processing apparatus may additionally include a scannerengine 107 configured to scan a document. A display unit 106 is used tomake setting in terms of various printing conditions such as a papertype, printing quality, etc. Reference numeral 108 denotes a userinterface such as a button or a touch panel. Reference numeral 109denotes an interface for a connection with an external apparatus such asa personal computer, a digital camera, etc. In the inside of theprinter, the parts described above are connected to each other via asystem bus.

Note that the printer may include other elements such as a power supplyunit, a feeder for handling paper, an interface for a direct connectionwith a network, etc. However, these elements are not essential to thepresent invention, and thus a further explanation thereof is omitted.

FIGS. 9 and 10 illustrate examples of printing environments to which thepresent embodiment of the invention is applicable. FIG. 9 illustrates anenvironment in which image data captured by a digital camera is printed.The image data captured by the digital camera 201 is stored in a memorycard 202. The memory card 202 is connected to the printer 101 and theimage data is printed by the printer 203. Instead of connecting thememory card 202 to the printer 203, the digital camera 201 may bedirectly connected to the printer 203 so that the image data can bedirectly transferred from the digital camera to the printer 203.

FIG. 10 illustrates an example of a printing environment including apersonal computer serving as a main apparatus. The personal computer 303has a hard disk for storing data acquired via various paths. Examples ofthe data stored on the hard disk include image data acquired from amemory card 301 via a card reader 302, image data downloaded from theInternet 306 via a router 305, etc. The data stored on the hard disk canbe printed by a printer 304 by operating the personal computer 303.

In the present embodiment, as described above, image data produced bythe image capturing apparatus 201 such as a digital camera is printed bythe image processing apparatus 101 such as a printer.

In the following explanation, by way of example, it is assumed that theimage capturing apparatus is a digital camera and the image processingapparatus is a printer. Note that the combination of apparatuses is notlimited to this example, but various other combinations are allowed. Forexample, a combination of a scanner and a printer, a combination of adigital camera and a monitor (an image processing application), or acombination of a monitor (an image processing application) and a printermay be used.

The image processing method according to the present embodiment isexecuted using an image processing apparatus 101 shown in FIG. 1. FIG. 1is a schematic diagram illustrating various processing units in theimage processing apparatus 101 according to the embodiment of thepresent invention.

In FIG. 1, image information D1 including an image of a subject capturedby the digital camera and also including associatedimage-capturing-condition information is stored in the image capturingapparatus 201 or the storage medium 202. The image capturing apparatus201 or the storage medium 202 is connected to an interface (I/F) 109serving as an interface of the image processing apparatus 101, and theimage information D1 is transferred to the image memory 104 via the I/F109. An image-capturing-condition information extraction unit 12extracts the image-capturing-condition information and the image datafrom the image information D1 stored in the image memory 104. Based onthe image-capturing-condition information extracted by theimage-capturing-condition information extraction unit 12, a processsetting unit 13 sets a process performed by a detection processing unit14. In accordance with the setting made by the process setting unit 13,the detection processing unit 14 performs a detection process on theimage data of the image information D1. A correction processing unit 15performs a correction process depending on a result of the detectionprocess performed by the detection processing unit 14 thereby generatingimage information D2. The resultant image information D2 is transmittedto a printer engine 105. The printer engine 105 produces an output byprinting the image information D2 on a printing medium.

Image Processing Apparatus According to the First Embodiment

An outline of a process performed by the image processing apparatusaccording to the first embodiment is described below with reference toFIG. 2.

FIG. 2 is a flow chart illustrating an operation procedure performed bythe image processing apparatus 101 according to the first embodiment. Aprogram corresponding to the flow chart shown in FIG. 2 is stored in theROM 103 or the RAM 104, and the program is read therefrom and executedby the CPU 102 thereby executing the process shown in the flow chart ofFIG. 2.

First, in step S21, the image processing apparatus 101 accesses, via theI/F 109, the image capturing apparatus 201 or the memory card serving asthe storage medium 202 and reads the image information D1 from thememory card into the image memory 104.

In the present example, it is assumed that the image capturing apparatus201 or the storage medium 202 is the memory card in which imageinformation D1 including image-capturing-condition information capturedby the digital camera has been stored. It is also assumed that theimage-capturing-condition information stored in the memory card includesinformation indicating the number of faces detected in the focal lengthadjustment process in an image capturing operation and also informationindicating directions, sizes, and positions of the respective detectedfaces.

Next, in step S22, the image-capturing-condition information extractionunit 12 analyzes the image information D1 stored in the image memory 104and extracts image-capturing-condition information indicating thenumber, directions, sizes, and positions of the detected faces.

Next, in step S23, based on the number, directions, sizes, and positionsof the faces indicated by the extracted image-capturing-conditioninformation, the process setting unit 13 performs setting associatedwith the face detection process to be performed by the detectionprocessing unit 14. Note that in the present embodiment, theimage-capturing-condition information is face detection information.

More specifically, in the setting associated with the face detectionprocess to be performed by the detection processing unit 14, thedirection, the size, and the position of the detector to be used in theface detection process are determined based on the number, directions,sizes, and positions of the faces described in theimage-capturing-condition information. Note that the detector refers toa template used to detect the specific area by matching. Although it isassumed in the present embodiment that the specific area is a face area,the specific area may be other parts included in the image. For example,the specific area may be a total figure of a person, a building, avehicle, etc.

Next, in step S24, the detection processing unit 14 performs the facedetection process in accordance with the setting performed by theprocess setting unit 13.

Next, in step S25, the correction processing unit 15 corrects the imagedata of the image information D1 in a manner depending on the result ofthe face detection process, and the correction processing unit 15outputs resultant corrected image information as image information D2.

Next, in step S26, the printer engine 105 performs printing on paper inaccordance with the corrected image information D2 produced by thecorrection processing unit 15.

In the conventional techniques, the above-described process, which isperformed in the present embodiment by the detection processing unit 14,is not performed but the result of the face detection performed by theimage capturing apparatus 201 is directly used by the correctionprocessing unit 15, or otherwise the correction processing unit 15 usesthe result of the face detection performed by the detection processingunit 14 without using image-capturing-condition information stored inthe image capturing apparatus 201 or the storage medium 202.

In the present embodiment, unlike the conventional techniques, based onthe face-detection-result information described in theimage-capturing-condition information stored in the image capturingapparatus 201 or the storage medium 202, the process setting unit 13sets parameters such as the direction, size, or position of the detectorused in the face detection performed by the detection processing unit 14as necessary.

The process in the example described above includes the correctionprocess on the input image information D1 and the printing process.However, in a case where only a face detection is necessary, thecorrection processing unit 15 and the printer engine 105 are notnecessary.

Although the memory card is used as the storage medium 202 in thepresent embodiment, other types of storage media may be used.

The image capturing apparatus 201 may be, for example, a digital cameraor a digital video camera. The image capturing apparatus 201 may be anapparatus having an image capturing unit and a connection unit whereby acaptured image can be stored in a built-in memory in the apparatus or ina memory card connected to the apparatus, and the stored image can betransferred to an external apparatus via the connection unit using aserial cable or the like such as a USB cable.

The image capturing apparatus 201 does not necessarily need to have astorage unit such as a hard disk if the image capturing apparatus 201has a built-in memory capable of storing image data includingimage-capturing-condition information and the stored image data can betransferred to an external apparatus via a serial cable or the like.

The storage medium 202 may be, for example, a memory card or a USBmemory. The image capturing apparatus 201 does not necessarily need tohave the storage medium 202 such as a memory card, if the imagecapturing apparatus 201 is configured such that image data includingimage data can be stored in a built-in memory and the image capturingapparatus 201 can be connected to an external apparatus.

The image information D1 is image data stored in a format such as Exif(Exchangeable image file format) that allows image-capturing-conditioninformation to be described within an image file.

Note that there is no particular restriction on the data format or thestorage format of the image information D1 as long as theimage-capturing-condition information is allowed to be related to theimage data.

That is, the image capturing apparatus 201 is an image capturingapparatus such as a digital camera and the storage medium 202 is astorage medium such as a memory card both configured to store imageinformation D1 of a captured image including image-capturing-conditioninformation.

Image Processing Unit According to the First Embodiment

The image processing unit 53 of the image processing apparatus 101according to the present embodiment of the invention is described indetail below with reference to drawings.

As shown in FIG. 1, the image processing unit 53 includes animage-capturing-condition information extraction unit 12, a processsetting unit 13, and a detection processing unit 14.

Image-Capturing-Condition Information Extraction Unit

The image-capturing-condition information extraction unit 12 is aprocessing unit configured to extract image-capturing-conditioninformation from the image information D1 stored in the image memory104.

In the present embodiment, by way of example, it is assumed that theimage information D1 is JPEG image data stored in the Exif format. FIG.3 illustrates a data structure of an Exif file. Main parts of the datastructure 30 of the Exif file are a header 31, a thumbnail 32, and mainimage data 33.

The main image data 33 is image data obtained by compressing capturedimage data in the JPEG (Joint Photographic Experts Group) format whichis one of still image data compression formats.

The thumbnail 32 is image data obtained by converting the main imagedata 33 into a size of 160×120.

The header 31 includes image-capturing-condition information indicatingconditions employed in the capturing of the main image. In the presentembodiment, it is assumed that the image-capturing-condition informationincludes face-detection-result information indicating a result of a facedetection performed in a focus adjustment process, in addition toinformation indicating the number of pixels, a compression format,date/time of capturing the image, etc. The face-detection-resultinformation includes information indicating the number of detectedfaces, the direction and the size of each detected face, and thecoordinate position in the captured image for each detected face.

The image-capturing-condition information extraction unit 12 checks theheader information of the image information stored in the image memory104 to determine whether the image information is stored in the Exiffile format. If so, the image-capturing-condition information extractionunit 12 extracts the face-detection-result information described in theExif header 31. The extracted face-detection-result information is sentto the process setting unit 13.

Process Setting Unit

The process setting unit 13 is a processing unit configured to performsetting associated with the detection process to be performed by thedetection processing unit 14, based on the image-capturing-conditioninformation extracted by the image-capturing-condition informationextraction unit 12.

More specifically, when the detection processing unit 14 is set toperform the face detection process, the process setting unit 13 sets thedirection of a template used in the face detection by the detectionprocessing unit 14, based on the face direction indicated by theface-detection-result information extracted by theimage-capturing-condition information extraction unit 12.

FIG. 4 illustrates a concept of a detector (template) 40 used in theface detection. A face included in a captured image is detected bymatching the captured image with the template 40. More specifically, thedetector (template) 40 shown in FIG. 4 is configured to detect faceelements such as a right eye 41, a left eye 42, a nose 43, and a mouth44 of a face. The face detection is performed by checking whether theright eye 41, the left eye 42, the nose 43, and the mouth 44 of thedetector 40 are all detected.

FIG. 5 is a diagram illustrating a concept of a face detection processin which the detector shown in FIG. 4 is used for eight directions. InFIG. 5, the eight directions of the detector 40 are set at angles of 0°,45°, 90°, 135°, 180°, 225°, 270°, and 315° where the reference directionof the detector is defined in an upward direction, and the angles aremeasured in a clockwise direction starting with 0° in the referencedirection.

In the conventional face detection technique, no information is providedas to the direction of a face included in image data of the imageinformation D1, and thus it is necessary to perform the templatematching for various angles of the detector 40 as shown in FIG. 5. Incontrast, in the present embodiment, the direction of the face as of thetime when the image was captured is detected from theface-detection-result information extracted by theimage-capturing-condition information extraction unit 12, and it isassumed that there is a high probability that a face in an input imageis in a similar direction to that of the face included in the capturedimage.

For example, in a case where the image-capturing-condition informationof the image information D1 includes information indicating that a faceis detected in a direction of 0°, i.e., in the upward direction in theimage data, it is expected that the face in the image data of the imageinformation D1 is in a direction of about 0°.

In this case in which the face is expected to be in the direction ofabout 0°, the detection processing unit 14 detects the face by checkingthe matching of the detector (template) with the image data for anglesof 0°, 45°, and 315° of the detector, in other words, by using threedetectors. FIG. 6 is a diagram illustrating a concept of settingperformed by the process setting unit 13 in terms of directions of thedetector employed in the face detection process performed by thedetection processing unit 14. In a case where the captured imageincludes a plurality of faces, the direction of the detector is set foreach of the plurality of faces so that all faces can be detected by theimage processing apparatus 101. For example, in a case where there is aface in a direction of about 0° and there is another face in a directionof about 180°, the detector 40 is set in six directions of 0°, 45°,135°, 180°, 225°, and 315°, and the face detection process is performedfor these six directions.

The face detection is performed, as described above, by detectingelements of the detector 40, i.e., the right eye 41, the left eye 42,the nose 43, and the mouth 44, and thus the detection process needs along time to detect so many elements. In a case where in the facedetection process, the detector is set in eight directions as shown inFIG. 5, the face detection needs a still longer processing time. In thepresent embodiment, to avoid the above problem, the directions in theface detection process are limited to three directions as shown in FIG.6 thereby achieving a reduction in the processing time.

In the present embodiment, the directions of the detector used in theface detection process by the detection processing unit 14 are set basedon the face directions described in the face-detection-resultinformation. However, the parameter of the detector set by the detectionprocessing unit 14 is not limited to the direction.

For example, the size of the detector used in the face detection processperformed by the detection processing unit 14 is set based on a facesize described in face-detection-result information.

FIG. 7 illustrates a concept of changing the size of the detector 40used in the face detection process. In the example shown in FIG. 7, thedetector 40 has a normal size, a detector 71 has a size one-half thenormal size in both vertical and horizontal direction, and a detector 72has a size twice the normal size in both vertical and horizontaldirections.

In the face detection process according to the conventional technique,no information is provided as to the size of a face in image data of theimage information D1, and thus it is necessary to use the detector 40for various sizes as shown in FIG. 7. In the present embodiment, incontrast, information on the size of a face in a captured image isprovided as a face-detection-result information extracted by theimage-capturing-condition information extraction unit 12, and thus it ispossible to expect that the size of a face in an input image is highlylikely to be close to the size described in the face-detection-resultinformation.

Thus, in the following explanation, by way of example, it is assumedthat the image-capturing-condition information of the image informationD1 includes information indicating that the size of the face in theimage data is close to one-half the normal size of the detector 40. Inthis case, it is expected that the face in the image data of the imageinformation D1 has a size close to one-half the normal size of thedetector 40.

If the face size is predicted to be nearly equal to one-half the normalsize of the detector 40, then the size of the detector used in the facedetection process by the detection processing unit 14 is set to one-halfthe normal size of the detector 40, and only the resultant detector 71set to this size is used in the face detection process.

As described above, the face detection is performed by detectingelements of the detector 40, i.e., the right eye 41, the left eye 42,the nose 43, and the mouth 44, and thus the detection process needs along time to detect so many elements. In a case where in the facedetection process, the detector is set to three sizes as shown in FIG.7, the face detection needs a still longer processing time. In thepresent embodiment, to avoid the above problem, the size of the detectoris limited to one size thereby achieving a reduction in the processingtime.

In a case where the captured image includes a plurality of faces withdifferent sizes, the detector is set to sizes corresponding to therespective sizes of the face so that all faces can be detected by theimage processing apparatus 101. For example, in a case where thecaptured image includes two faces one of which has a size close toone-half the normal size of the detector 40 and the other one of whichhas a size close to twice the normal size of the detector 40, the facedetection process is performed using two detectors, i.e., the detector71 with the size one-half the normal size of the detector 40 and thedetector 72 with the size twice the normal size of the detector 40.

Detection Processing Unit

The detection processing unit 14 is a processing unit configured toperform the detection process in accordance with the setting performedby the process setting unit 13.

The detection processing unit 14 performs the detection process such asthe face detection process. The face detection process may be performedin various manners. In the present embodiment, the detector 40 fordetecting the right eye 41, the left eye 42, the nose 43, and the mouth44 is used, and a check is made for various angles and sizes of thedetector 40 as to whether the right eye 41, the left eye 42, the nose43, and the mouth 44 are detected in the image data of the imageinformation D1.

For example, the process setting unit 13 may set the detection processsuch that the detection is performed for only three angles around 0°,i.e., 0°, 45°, and 315°, and for only one size, using the detector 71shown in FIG. 7 having the size that is one-half the normal size of thedetector 40. In this case, the image data of the image information D1 issubjected to the face detection process in which the detector 71 withthe size one-half the normal size of the detector 40 is used and thecaptured image is compared with the detector 71 in three directions of0°, i.e., 0°, 45°, and 315°.

In the conventional face detection technique, no information is providedas to the direction of a face in image data of the image information D1,and thus it is necessary to perform the detection process using threedetectors having different sizes such as those shown in FIG. 7 in eightdifferent directions such as those shown in FIG. 5. That is, in thiscase, it is necessary to perform the face direction process for a totalof 24 detection conditions. In the present example, the detector 71 withthe size one-half the normal size of the detector 40 is used, and thedetection process is performed only in three directions of 0°, i.e., 0°,45°, and 315°. Thus, the detection process is performed only for threedetection conditions. The reduction in the number of processingconditions leads to a reduction in processing time.

The detection process performed by the detection processing unit 14 issimilar to the detection process that is performed to produce theimage-capturing-condition information stored in the image capturingapparatus 201 or the storage medium 202. There is no particularrestriction on the details of the detection process as long as theparameters of the detection process performed by the detectionprocessing unit 14 can be limited based on the image-capturing-conditioninformation.

In the present embodiment, it is assumed that the image capturingapparatus 201 is a digital camera or the like and the image processingapparatus 101 is a printer or the like. The present embodiment is alsoapplicable to a combination of a scanner and a printer. In this case,the scanner and the printer both have a detection processing unitconfigured to detect a specific area, and the printer performs adetection process based on information obtained as a result of adetection process performed by the scanner. The present embodiment isalso applicable to a combination of a digital camera and a personalcomputer (PC) connected to the digital camera. In this case, the PC mayperform a detection process based on information obtained as a result ofa detection process performed by the digital camera. The presentembodiment is also applicable to a combination of a PC (PersonalComputer) and a printer connected to the PC. In this case, the printermay perform a detection process based on information obtained as aresult of a detection process performed by the PC.

In the first embodiment, as described above, when the external apparatusand the image processing apparatus both have the capability of detectinga specific area, the image processing apparatus performs the process ofdetecting the specific area based on the information obtained as aresult of the detection process performed by the external apparatusthereby allowing the image processing apparatus to have the improvedperformance and to perform the detection process in a shorter processingtime.

Second Embodiment

An outline of a process performed by an image processing apparatusaccording to a second embodiment is described below.

In the first embodiment described above, when the image processingapparatus performs a detection process similar to that performed by theimage capturing apparatus such as a digital camera, the detection resultstored in the image capturing apparatus is used in the detection processperformed by the image processing apparatus thereby achievingsimplification of the process. However, in the first embodiment, thereis a possibility that there is a specific area that cannot be detectedby the image capturing apparatus, depending on the performance of theimage capturing apparatus. In the detection process performed by theimage processing apparatus, it is impossible to detect such a specificarea that cannot be detected by the image capturing apparatus. This isbecause the specific area detected in the detection process by the imageprocessing apparatus can be specified only by the detection resultstored in the image capturing apparatus. That is, the angles or sizes ofthe template used in the detection process performed by the imagecapturing apparatus are limited to specific values, and thus a specificarea with an angle or size different from the limited values cannot bedetected by the image processing apparatus. In the second embodiment,the problem described above is handled.

In the second embodiment, the image processing apparatus 101 configuredas shown in FIG. 1 is also used as in the first embodiment describedabove. However, the second embodiment is different from the firstembodiment in that the processes performed by the process setting unit13 and the detection processing unit 14 shown in FIG. 1 are differentfrom those according to the first embodiment. In the followingexplanation of the second embodiment, a description of similar processesto those in the first embodiment is omitted.

The operation procedure performed by the image processing apparatus 101is similar to that according to the first embodiment described abovewith reference to the flow chart shown in FIG. 2, and thus a duplicatedexplanation of the operation procedure is omitted.

Image Processing Unit According to Second Embodiment

The image processing unit 53 of the image processing apparatus accordingto the present embodiment of the invention is described in detail belowwith reference to drawings.

As shown in FIG. 1, the image processing unit 53 includes animage-capturing-condition information extraction unit 12, a processsetting unit 13, and a detection processing unit 14. In the presentembodiment, the image-capturing-condition information extraction unit 12shown in FIG. 1 is similar to that according to the first embodiment,and thus a further description thereof is omitted, and the descriptionis focused on the process performed by the process setting unit 13 andthat performed by the detection processing unit 14.

Process Setting Unit

The process setting unit 13 is a processing unit configured to performsetting associated with the detection process to be performed by thedetection processing unit 14, based on the image-capturing-conditioninformation extracted by the image-capturing-condition informationextraction unit 12.

In the following explanation, by way of example, it is assumed that thedetection processing unit 14 performs the face detection process. Basedon the face direction indicated in the face-detection-result informationextracted by the image-capturing-condition information extraction unit12, the process setting unit 13 sets the weight for the direction of thedetector used in the face detection process performed by the detectionprocessing unit 14.

In the face detection process according to the conventional technique,no information is provided as to the direction of a face included inimage data of the image information D1, and thus it is necessary to usethe detector 40 for various angles assigned the same weight, as shown inFIG. 5. In the present embodiment, in contrast, information on thedirection of a face in a captured image is provided as aface-detection-result information extracted by theimage-capturing-condition information extraction unit 12, and thus it ispossible to expect that the direction of a face in an input image ishighly likely to be close to the direction described in theface-detection-result information. Furthermore, as for faces that werenot detected by the image capturing apparatus such as a digital camera,it is also expected that these undetected faces are also highly likelyto be in the same direction as the direction of the detected face.

In this case in which the face is expected to be in the direction ofabout 0°, a high weight is assigned to angles around the angle of 0°,i.e., 0°, 45°, and 315° for the detector used in the face detectionprocess performed by the detection processing unit 14. On the otherhand, a lower weight is assigned to other angles, i.e., 90°, 135°, 180°,225°, and 270°.

In the face detection, as described above, if the detector used in theface detection is set to eight directions as shown in FIG. 5, the weightis set to be equal for all directions in some of which there may be noface. This leads to an increase in probability of making an erroneousdetection. In the present embodiment, in view of the above, the weightsof the detectors used by the detection processing unit 14 are changeddepending on the face direction indicated in the face-detection-resultinformation extracted by the image-capturing-condition informationextraction unit 12 thereby reducing the probability of making anerroneous detection.

In the example described above, the detection processing unit 14 setsthe weights of the directions of the detector used in the face detectionprocess performed by the detection processing unit 14, based on the facedirections described in the face-detection-result information. However,the parameter of the detector for which the weight is set by thedetection processing unit 14 is not limited to the direction.

For example, based on a face size described in face-detection-resultinformation, the weight may be defined for the size of the detector usedin the face detection performed by the detection processing unit 14.

In the conventional techniques, no information is provided as to thesize of a face in image data of the image information D1, and thus it isnecessary to use the detector 40 for various sizes such as those shownin FIG. 7 assigned the same weight. In the present embodiment, incontrast, information on the size of a face in a captured image isprovided as a face-detection-result information extracted by theimage-capturing-condition information extraction unit 12, and thus it ispossible to expect that the size of a face in an input image is highlylikely to be close to the size described in the face-detection-resultinformation.

Thus, in the following explanation, by way of example, it is assumedthat the image-capturing-condition information of the image informationD1 includes information indicating that the size of the face in theimage data is close to one-half the normal size of the detector 40. Inthis case, it is expected that the face in the image data of the imageinformation D1 actually has a size close to one-half the normal size ofthe detector 40.

If the face size is predicted to be nearly equal to one-half the normalsize of the detector 40, and thus the weight is assigned to the detectorused by the detection processing unit 14 such that the detector 71 withthe size one-half the normal size of the detector 40 has a high weightbut other detectors have a lower weight.

In the face detection process, as described above, if the same weight isassigned to three sizes shown in FIG. 7 of the detectors used in theface detection process, then even sizes that are very different from theactual face size have the same weight. This leads to an increase inprobability of making an erroneous detection. In the present embodiment,in view of the above, the weights of the detectors are changed dependingon the face size indicated in the face-detection-result informationextracted by the image-capturing-condition information extraction unit12 thereby reducing the probability of making an erroneous detection.

Detection Processing Unit

The detection processing unit 14 is a processing unit configured toperform the detection process in accordance with the setting performedby the process setting unit 13.

In the following explanation, by way of example, it is assumed that thedetection processing unit 14 performs the face detection process. Theface detection process may be performed in various manners. In thepresent embodiment, the detector (template) 40 for detecting the righteye 41, the left eye 42, the nose 43, and the mouth 44 is used, and thedetermination is made as to whether the right eye 41, the left eye 42,the nose 43, and the mouth 44 are detected in the image data of the ofthe image information D1.

For example, the process setting unit 13 may set the detection processsuch that a high weight is assigned to angles around the angle of 0°,i.e., 0°, 45°, and 315° for the detector shown in FIG. 5 while a lowerweight is assigned to other angles, i.e., 90°, 135°, 180°, 225°, and270°, and furthermore the weights of the sizes of the detectors shown inFIG. 7 may be set such that the detector 71 with the size one-half thenormal size of the detector 40 has a high weight while other detectorshave a lower weight.

The face detection process is performed for all angles of the template,i.e., 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°. A threshold valueis set for the matching level in the face detection process, and thethreshold value is changed depending on the weight. For example, forangles with weights greater than a first predetermined value, thethreshold value is set to be smaller than a second predetermined value,while the threshold value is set to be greater than the secondpredetermined value for angles with weights smaller than the firstpredetermined value. This results in a reduction in probability ofdetecting a face in an image corresponding to a template with an anglehaving a low weight. The reduction in probability of detecting such aface leads to a reduction in the number of faces to be subjected to thecorrection process performed by the correction processing unit, whichleads to a reduction in processing load, which are advantages obtained.As described above, the threshold value used in the face detectionprocess is set for each angle of the template.

As for the sizes, the threshold value is set to be small for sizes withhigh weights, while the threshold value is set to be large for sizeswith low weights.

In the face detection process performed by the detection processing unit14, the detector 40 is used to detect face elements such as the righteye 41, the left eye 42, the nose 43, and the mouth 44. In the detectionof the face elements, matching scores may be defined. For example, ascore of 100 is given to the matching of each of the right eye 41 andthe left eye 42, 50 to the nose 43, and 70 to the mouth 44. Furthermore,the total matching score of the detector 40 is given such that thematching is evaluated for each element, the matching score is given fora detected element, and the total matching score is given by the sum ofscores for the respective detected elements. A threshold value is setfor the total matching score. For example, when the threshold value isset to 250, if the total matching score is greater than this thresholdvalue, then it is determined that a face is detected.

For example, if the right eye 41, the left eye 42, the nose 43, and themouth 44 are all detected in a particular area, then the total matchingscore of the detector 40 for this area is given as 100+100+50+70=320. Inthis case, the total matching score is greater than the threshold value,i.e., 250, and thus it is determined that a face is detected. On theother hand, in a case where the right eye 41, the nose 43, and the mouth44 are detected in a particular area, the total matching score is givenas 100+50+70=220. In this case, the matching score is smaller than thethreshold value, i.e., 250, and thus it is determined that there is noface in this specific area.

In the conventional face detection technique, no information is providedas to the direction and the size of a face in image data of the imageinformation D1. Therefore, in the face detection process, the sameweight is given to even those directions in which there is no face oreven to those sizes that are very different from the actual size. Thisleads to an increase in probability of making an erroneous detection. Inthe present embodiment, in contrast, the weight is set to the detectionprocess performed by the detection processing unit 14 based on thedetection result described in the image-capturing-condition informationthereby achieving a reduction in the probability of making an erroneousdetection.

In the first embodiment, in the case where there is a specific area thatcannot be detected by the image capturing apparatus such as a digitalcamera due to the performance of the image capturing apparatus, thespecific area cannot be detected by the image processing apparatus.

In the second embodiment, the weight for the detection process performedby the detection processing unit 14 is set by the process setting unit13 based on the extracted image-capturing-condition information, and thedetection process is performed for the entire area of the image data ofthe image information D1 thereby making it possible to detect even thespecific area undetectable by the image capturing apparatus andachieving a reduction in probability of making an erroneous detection.

In the second embodiment, as described above, when the externalapparatus and the image processing apparatus both have the capability ofdetecting a specific area, the weight for the angle or the size of thetemplate used in the detection process performed by the image processingapparatus is set based on the extracted image-capturing-conditioninformation thereby making it possible to detect even the specific areaundetectable by the image capturing apparatus and thus achieving animprovement in detection performance.

Third Embodiment

The present invention may be applied to a system including a pluralityof devices (such as a host computer, an interface device, a reader, aprinter, etc.) or may be applied to an apparatus including only a singledevice (such as a copy machine, a facsimile machine, etc.).

One or more of functions according to one of or a mixture of theembodiments described above may be implemented by loading a softwareprogram code from a medium onto a computer (or a CPU or an MPU) of asystem or an apparatus, and executing the program on the computer. Inthis case, the program code read from a computer-readable storage mediumimplements the novel functions disclosed in the embodiments describedabove, and the storage medium on which the program code is stored fallswithin the scope of the present invention. Specific examples of storagemedia for supplying the program code include a floppy (registeredtrademark) disk, a hard disk, an optical disk, a magneto-optical disk, aDVD-ROM disk, a DVD-ROM disk, a DVD-R disk, a CD-ROM disk, a CD-R disk,a magnetic tape, and a nonvolatile memory card.

The functions disclosed in the embodiments may be implemented not onlyby executing the program code on a computer, but part or all of theprocess may be performed by an operating system or the like running onthe computer in accordance with the program code. Such implementation ofthe functions also falls within the scope of the present invention.

The program code may be read from a storage medium and loaded into afunction extension board inserted in a computer or into a functionextension unit connected to the computer thereby realizing one or morefunctions according to any embodiment of the invention described above.In this case, more specifically, the program code read from the storagemedium is written in a memory disposed on the function extension boardor the function extension unit. and a CPU or the like disposed in thefunction extension board or the function extension unit may perform partor all of the process according to the loaded program code therebyachieving one or more functions according to any embodiment of theinvention.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all modifications and equivalent structures and functions.

This application claims the benefit of Japanese Patent Application No.2008-162301 filed Jun. 20, 2008, which is hereby incorporated byreference herein in its entirety.

1. An image processing apparatus comprising: an input unit configured toinput, from an external apparatus, image data andimage-capturing-condition information obtained by an image capturingoperation; a determining unit configured to determine an angle or a sizeof a template used in detecting a specific area from the image data,based on the image-capturing-condition information input via the inputunit; and a detection unit configured to detect a specific area from theimage data by comparing the image data with the template having theangle or the size determined by the determining unit, wherein theexternal apparatus and the image processing apparatus are bothconfigured to detect the specific area, analyze the image data, whereinthe image processing apparatus receives the image-capturing-conditioninformation as a result of a first detection process that has alreadybeen performed by the external process, and wherein the image processingapparatus performs a second detection process based on theimage-capturing-condition information derived from the first detectionprocess.
 2. The image processing apparatus according to claim 1,wherein: the image-capturing-condition information is face detectioninformation obtained by an image capturing operation; and the specificarea is a face area.
 3. The image processing apparatus according toclaim 2, wherein the face detection information is informationindicating a face direction or a face size.
 4. The image processingapparatus according to claim 1, wherein: the determining unitdetermines, based on the image-capturing-condition information, a weightfor each angle of the template used in detecting the specific area; andthe detection unit sets, based on the weight, a threshold value fordetecting the specific area for angle of the template, and the detectionunit compares the specific area included in the image data with thetemplate corresponding to an angle of the specific area to detect thespecific area by determining whether a matching degree in the comparisonis greater than the threshold value.
 5. The image processing apparatusaccording to claim 4, wherein when the weight is greater than a firstpredetermined value, the threshold value is set to be smaller than asecond predetermined value, while the threshold value is set to begreater than the second predetermined value when the weight is smallerthan the first predetermined value.
 6. The image processing apparatusaccording to claim 4, wherein when the matching degree in the comparisonis greater than the threshold value, the specific area is detected. 7.An image processing method performed in an image processing apparatus,the method comprising the steps of: inputting, from an externalapparatus, image data and image-capturing-condition information obtainedby an image capturing operation; determining an angle or a size of atemplate used in detecting a specific area from the image data, based onthe image-capturing-condition information input in the inputting step;and detecting a specific area by comparing the image data with thetemplate having the angle or the size determined in the determiningstep, wherein the external apparatus and the image processing apparatusare both configured to detect the specific area, wherein the imageprocessing apparatus receives the image-capturing-condition informationas a result of a first detection process that has already been performedby the external process, and wherein the image processing apparatusperforms a second detection process based on theimage-capturing-condition information derived from the first detectionprocess.
 8. The image processing apparatus according to claim 7,wherein: the image-capturing-condition information is face detectioninformation obtained by an image capturing operation; and the specificarea is a face area.
 9. A control program storable in non-transitory astorage medium readable by a computer, the control problem being adaptedto make the computer execute an image processing method, the imageprocessing method executed by the computer comprising the steps of:inputting, from an external apparatus, image data andimage-capturing-condition information obtained by an image capturingoperation; determining an angle or a size of a template used indetecting a specific area from the image data, based on theimage-capturing-condition information input in the inputting step; anddetecting a specific area from the image data by comparing the imagedata with the template having the angle or the size determined in thedetermining step, wherein the external apparatus and the imageprocessing apparatus are both configured to detect the specific area,wherein the image processing apparatus receives theimage-capturing-condition information as a result of a first detectionprocess that has already been performed by the external process, andwherein the image processing apparatus performs a second detectionprocess based on the image-capturing-condition information derived fromthe first detection process.
 10. An image processing apparatuscomprising: an input unit configured to input, from an externalapparatus, image data and image-capturing-condition information obtainedby an image capturing operation; a determining unit configured todetermine an angle or a size of a template used in detecting a specificarea from the image data, based on the image-capturing-conditioninformation input via the input unit; and a detection unit configured todetect a specific area from the image data by comparing the image datawith the template having the angle or the size determined by thedetermining unit wherein the determining unit determines, based on theimage-capturing-condition information, a weight for each angle of thetemplate used in detecting the specific area; and the detection unitsets, based on the weight, a threshold value for detecting the specificarea for angle of the template, and the detection unit compares thespecific area included in the image data with the template correspondingto an angle of the specific area to detect the specific area bydetermining whether a matching degree in the comparison is greater thanthe threshold value.
 11. The image processing apparatus according toclaim 10, wherein when the weight is greater than a first predeterminedvalue, the threshold value is set to be smaller than a secondpredetermined value, while the threshold value is set to be greater thanthe second predetermined value when the weight is smaller than the firstpredetermined value.
 12. The image processing apparatus according toclaim 10, wherein when the matching degree in the comparison is greaterthan the threshold value, the specific area is detected.
 13. An imageprocessing method performed between an external apparatus and an imageprocessing apparatus, wherein the external apparatus and the imageprocessing apparatus are both configured to detect a specific area froman image, analyze image data, and derive image-capturing-conditioninformation, the method comprising the steps of: performing a firstdetection process in the external apparatus and derivingimage-capturing-condition information by analyzing the image data;inputting into the image processing apparatus, from the externalapparatus, the image data and image-capturing-condition informationobtained by an image operation in the external process via the firstdetection process; determining in the image processing apparatus anangle or a size of a template used in detecting a specific area from theimage data, based on the image-capturing-condition information input inthe inputting step; and performing a second detection process in theimage processing apparatus by detecting a specific area and comparingthe image data with the template having the angle or the size determinedin the determining step, wherein the image processing apparatus performsthe second detection process based on the image-capturing-conditionderived from the first detection process.